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GRANT  TITLE:  Impact  of  Computer-Mediated  Communication  Media  Characteristics  on  Information 
Acquisition,  Attitude  Favorability,  and  Intentions  toward  Joining  the  Navy 

AWARD  PERIOD:  1  July  2001  -  30  September  2002 

OBJECTIVE:  To  investigate  the  extent  to  which  characteristics  of  computer-mediated  communication 
media  (CMCM)  are  associated  with  a  receiver’s  communication  outcomes. 

APPROACH:  A  set  of  forty  web  sites  was  randomly  selected  from  the  Fortune  500  list  of  companies  for 
use  by  two  sets  of  college  students.  The  first  group,  representing  IT  experts,  examined  objective 
characteristics  of  the  web  sites  (e.g.,  amount  of  information,  hyperlink  functionality).  This  group  included 
approximately  40  graduate  students  enrolled  in  an  advanced  course  in  information  systems  analysis  and 
design.  Each  member  of  the  expert  group  was  assigned  at  random  to  judge  the  attributes  of  five  web  sites 
selected  from  the  list  of  forty  sites.  Five  experts  judged  each  web  site,  but  no  two  experts  were  allowed  to 
judge  the  same  set  of  five  web  sites.  The  order  in  which  web  sites  appeared  on  the  lists  of  the  five  experts 
was  varied  to  control  for  biases  caused  by  the  order  of  presentation. 

The  second  group  examined  subjective  characteristics  of  the  web  sites  (e.g.,  social  presence,  perceived 
richness,  realism).  This  group,  representing  non-experts  (or  typical  visitors  to  a  web  site),  consisted  of 
approximately  400  students  enrolled  in  initial  undergraduate  courses  intended  to  familiarize  them  with 
standard  office  software  commonly  available  on  personal  computers.  Undergraduate  students  with 
extensive  IT  experience  were  not  allowed  to  participate  in  the  study.  Each  non-expert  was  asked  to  navigate 
one  of  the  forty  websites,  which  was  randomly  assigned. 

Instruments  developed  for  both  experts  and  non-experts  were  focused  primarily  on  perceptions  of  web  site 
outcomes  and  three  types  of  web  site  attributes:  content,  presentation,  and  convenience.  Ease-of-use  and 
usefulness  were  also  measured,  as  were  the  respondent's  previous  experience  with  Internet  web  sites  and 
their  perceived  self-efficacy  for  using  the  Internet  to  accomplish  their  goals. 

Differences  between  experts’  and  non-experts’  perceptions  and  evaluations  of  web  pages  were  analyzed 
using  ANOVA.  For  each  dependent  variable,  separate  ANOVAs  were  used  to  compare  the  average  expert 
ratings  with  the  average  ratings  of  non-experts  across  the  forty  web  sites. 

Relationships  between  web  site  characteristics  and  receiver  reactions  were  assessed  as  follows.  First,  the 
group  of  400  non-experts  (ten  per  web  site)  was  divided  randomly  into  two  groups  of  200  non-experts  (five 
per  web  site)  to  mitigate  most  of  the  negative  affects  of  common  method  variance.  For  convenience,  the 
two  non-expert  groups  are  labeled  "A"  and  "B."  Responses  obtained  from  group  "A"  provided  the  criterion 
data  (e.g.,  measures  of  usability,  ease  of  use,  usefulness,  effectiveness,  and  satisfaction)  for  the  forty  web 
sites.  Average  attribute  ratings  for  the  web  site  characteristics  (predictor  variables  reflecting  content, 
presentation,  and  convenience)  were  obtained  from  group  "B"  and  the  experts.  Hierarchical  regression 
analysis  is  used  to  test  the  relative  ability  of  attribute  ratings  made  by  experts  and  non-experts  (from  group 
"B"  only)  to  explain  variance  in  criterion  measures  obtained  from  non-experts  in  group  "A." 

ACCOMPLISHMENTS  (throughout  award  period):  The  first  accomplishment  was  the  development  of  an 
integrated  model  of  the  influence  of  CMCM  characteristics  on  receiver  attitudes,  intentions,  and  behaviors 
(Figure  1).  The  left  portion  of  the  model  is  based  on  genre  theoiy  (Yates  and  Orlikowski,  1992).  Content 
consists  of  qualities  of  information  contained  in  the  web  site  (e.g.,  amount,  type,  complexity,  diversity, 


specificity,  uniqueness,  and  realism).  Design  consists  of  channel  capacities  (cue  types)  search  capabilities, 
availability  of  two-way  communication,  interactiveness,  convenience,  and  attractiveness.  Individual 
differences  include  demographic  variables  such  as  gender,  race,  age,  and  education. 

The  right  portion  of  the  model  is  based  on  the  Theories  of  Reasoned  Action  and  Planned  Behavior  (Ajzen 
and  Fishbein,  1980;  Ajzen,  1991).  Receivers’  communication  outcomes  include  perceptions  (e.g.,  symbolic 
meaning  and  social  presence),  reactions  (e.g.,  involvement  and  engagement),  media  judgments  (e.g., 
useability,  usefulness,  and  ease-of-use),  and  apprehension  (e.g.,  recall,  comprehension,  ambiguity,  and 
uncertainty).  Effectiveness  outcomes  include  attitudes  toward  communication  (e.g.,  toward  organization 
and  pursuing  employment),  intentions  (e.g.,  to  acquire  additional  information,  to  return  to  web  site,  and  to 
join  organization),  behavior  (e.g.,  use  of  FAQ  or  e-mail),  and  receivers’  overall  judgments  (e.g., 
effectiveness  of  web  site,  satisfaction  with  web  site,  and  impressions). 

The  second  accomplishment  of  the  project  was  an  examination  of  the  relationships  between  CMCM 
characteristics  and  communication  receiver  outcomes  described  in  Figure  1.  Data  describing  those 
relationships  were  drawn  from  surveys  of  information  systems  experts  and  non-experts.  Questions  specific 
to  non-experts  allow  an  examination  of  media  features  on  communication  outcomes  (Pitts  et  al.,  in 
progress),  while  questions  directed  at  both  groups  allow  a  comparison  of  differences  in  perceptions  of  web 
site  characteristics  by  IT  experts  and  non-experts  (Van  Scotter  et  al.,  2002;  Van  Scotter  et  al.,  work  in 
progress). 

The  Role  of  Content,  Form,  and  Functionality  on  Web  Site  Effectiveness.  Results  from  this  portion  of  the 
study  (Pitts  et  al.,  work  in  progress)  support  the  existence  of  generally  expected  relationships  predicted  by 
genre-oriented  research.  The  results  also  infer  the  mediating  role  of  perceptions  of  usefulness  and  ease-of- 
use  between  message  content  and  form  on  the  one  hand,  and  web  site  effectiveness  on  the  other. 

Preliminary  results  demonstrate  the  importance  of  message  content  and  form  in  explaining  perceived 
usefulness  and  perceived  ease-of-use  constructs  because  they  show  the  importance  of  inputs  (rather  than 
simply  the  processing  mechanisms)  to  technology  acceptance.  For  example,  the  importance  of  message 
content  and  form  over  media  characteristics  is  supported  by  the  low  A  adjusted  R2  values  between  web- 
browser  self-effectiveness  and  the  receiver  variables  of  perceived  usefulness,  ease-of-use,  and 
communication  effectiveness. 

Differences  in  perceptions  of  web  site  characteristics  by  IT  experts  and  non-experts.  This  portion  of  the 
study  (Van  Scotter  et  al.,  2002;  Van  Scotter  et  al.,  work  in  progress)  is  primarily  exploratory  in  nature.  The 


perceptions  of  three  groups  of  respondents  (experts,  and  two  groups  of  non-experts)  are  compared  to 
eliminate  most  of  the  negative  affects  of  common  method  variance.  Preliminary  results  suggest  that  users 
focus  more  on  content  because  it  is  more  closely  related  to  their  purpose  for  visiting  a  web  site  while  IT 
experts  care  more  about  design  features  (i.e.,  process-related  aspects).  Preliminary  results  also  suggest  that 
amount  of  information  and  perceived  complexity  explain  more  variance  in  user  satisfaction  than  IT  expert 
satisfaction. 

SIGNIFICANCE: 

The  Role  of  Content,  Form,  and  Functionality  on  Web  Site  Effectiveness .  Results  from  this  portion  of  the 
study  (Pitts  et  al.,  in  progress)  provides  an  important  extension  of  the  Technology  Acceptance  Model 
(Davis,  1989)  by  showing  that,  in  the  case  of  web-based  communication,  perceived  usefulness  and  ease-of- 
use  are  closely  associated  with  variance  in  message  content  and  form.  This  finding  suggests  that 
information  systems  researchers  must  pay  more  attention  to  what  is  said  in  the  web  site  and  how  it  is  said, 
rather  than  what  medium  is  used  to  say  it.  The  study  also  shows  the  usefulness  of  genre  research  in 
conceptualizing  past  IS  research  (which  has  traditionally  focused  on  underlying  technologies)  into  a  way 
that  technologies  are  being  used  (e.g.,  how  message  content  and  form  interacts  with  technology  and  the 
receiving  audience). 

Differences  in  perceptions  of  web  site  characteristics  by  IT  experts  and  non-experts.  The  results  of  this 
portion  of  the  study  (Van  Scotter  et  al.,  2002;  Van  Scotter  et  al.,  work  in  progress)  reveal  the  types  of 
website  characteristics  that  have  significant  effects  on  perceptions  of  usefulness,  usability,  ease  of  use 
satisfaction  and  effectiveness  judgments  of  non-expert  users.  In  addition,  the  study  evaluates  the  relative 
influence  of  web  site  designers’  and  users’  perceptions  of  web  site  attributes  on  judgments  of  web  site 
usefulness,  ease  of  use,  satisfaction  and  effectiveness  made  by  non-expert  users.  On-going  analysis  may 
show  that  the  perceptions  of  IT  experts  are:  1)  different  and  relevant;  2)  different,  but  not  relevant;  or  3)  not 
different  from  the  perceptions  of  non-experts.  If  the  results  support  the  first  situation,  then  both  IT  experts 
and  non-experts  can  provide  valuable  data  to  inform  web  site  design.  If  results  support  situation  2  or  3, 
designers  are  probably  better  off  listening  to  users  when  it  comes  to  designing  interfaces.  The  more 
variance  in  outcomes  explained  by  non-experts  perceptions,  the  more  important  it  is  for  web  site  designers 
to  solicit  their  opinions  and  to  allow  users  views  to  guide  design  goals  and  the  design  process.  This  study 
extends  previous  work  showing  that  differences  in  experience  and  purposes  for  using  information  are 
important  predictors  of  effectiveness  and  other  outcomes.  Future  research  should  focus  on  sets  of  predictor 
variables  that  make  a  significant  difference  in  these  dependent  variables. 
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